Видео с ютуба Sparse Transformer

Giannis Daras: Improving sparse transformer models for efficient self-attention (spaCy IRL 2019)

Giannis Daras: Improving sparse transformer models for efficient self-attention (spaCy IRL 2019)
Объяснение принципа разреженного внимания DeepSeek: на 80% дешевле ИИ с длинным контекстом

Объяснение принципа разреженного внимания DeepSeek: на 80% дешевле ИИ с длинным контекстом
Sparse Transformers and MuseNet | AISC

Sparse Transformers and MuseNet | AISC
Revolutionizing AI: The Sparse Transformer Explained!

Revolutionizing AI: The Sparse Transformer Explained!
What are Sparse Transformers?

What are Sparse Transformers?
BigBird Research Ep. 3 - Block Sparse Attention, ITC vs. ETC

BigBird Research Ep. 3 - Block Sparse Attention, ITC vs. ETC
A Window Into LLMs | Sparse Autoencoders Explained

A Window Into LLMs | Sparse Autoencoders Explained
Как внимание стало настолько эффективным [GQA/MLA/DSA]

Как внимание стало настолько эффективным [GQA/MLA/DSA]
Weight-sparse transformers have interpretable circuits (OpenAI Research)

Weight-sparse transformers have interpretable circuits (OpenAI Research)
BigBird Transformer за 3 минуты! | Разреженное внимание, теория графов и линейные трансформеры

BigBird Transformer за 3 минуты! | Разреженное внимание, теория графов и линейные трансформеры
Sparse Autoencoders: Progress & Limitations with Joshua Engels

Sparse Autoencoders: Progress & Limitations with Joshua Engels
Attention in transformers, step-by-step | Deep Learning Chapter 6

Attention in transformers, step-by-step | Deep Learning Chapter 6
Soft Mixture of Experts - An Efficient Sparse Transformer

Soft Mixture of Experts - An Efficient Sparse Transformer
Blur-aware Spatio-temporal Sparse Transformer for Video Deblurring (CVPR 24)

Blur-aware Spatio-temporal Sparse Transformer for Video Deblurring (CVPR 24)
Sparse is Enough in Scaling Transformers (aka Terraformer) | ML Research Paper Explained

Sparse is Enough in Scaling Transformers (aka Terraformer) | ML Research Paper Explained
Embracing Single Stride 3D Object Detector with Sparse Transformer

Embracing Single Stride 3D Object Detector with Sparse Transformer
Sparse LLMs at inference: 6x faster transformers! | DEJAVU paper explained

Sparse LLMs at inference: 6x faster transformers! | DEJAVU paper explained
CVPR2023 Sparsifiner: Learning Sparse Instance-Dependent Attention for Efficient Vision Transformers

CVPR2023 Sparsifiner: Learning Sparse Instance-Dependent Attention for Efficient Vision Transformers
Is Sparse Attention more Interpretable?

Is Sparse Attention more Interpretable?